Create tracked types to handle inactive input sources by nvddr · Pull Request #167 · NVIDIA/IsaacTeleop

nvddr · 2026-02-24T18:34:44Z

Summary by CodeRabbit

API Changes
- Tracker queries (hands, head, controllers, pedals, full-body) now return tracked-wrapper objects whose .data is present when active; explicit is_active fields removed. Python/device IO tensor types and bindings updated to tracked variants and optional-data semantics.
Impact
- Check .data presence before use. Tests, samples, and integrations updated to the new tracked-data access patterns; runtime behavior unchanged when data is present.

github-actions · 2026-02-24T18:35:31Z

Docs preview

Built docs for this PR are available at:

https://NVIDIA.github.io/IsaacTeleop/preview/pr-167/

(It may take a minute to appear after the workflow completes.)

github-actions · 2026-02-24T18:42:47Z

Docs preview

Built docs for this PR are available at:

https://NVIDIA.github.io/IsaacTeleop/preview/pr-167/

(It may take a minute to appear after the workflow completes.)

Copilot

Pull request overview

This PR updates the DeviceIO tracking API to represent inactive input sources via new *TrackedT wrapper types (with a nullable .data), and aligns schema/types/tests/examples with that model (including renaming pedal output validity to is_active).

Changes:

Add *Tracked FlatBuffers tables and corresponding Python bindings that expose a nullable .data payload when a source is inactive.
Update DeviceIO C++ trackers + pybind APIs to return tracked wrappers (and remove prior is_active fields where applicable).
Update plugins, examples, and schema tests to use the new tracked-wrapper pattern and renamed pedal field.

Reviewed changes

Copilot reviewed 57 out of 57 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
src/plugins/manus/core/manus_hand_tracking_plugin.cpp	Switch controller access to tracked wrapper `.data` checks for injection timing/poses.
src/plugins/generic_3axis_pedal/generic_3axis_pedal_plugin.cpp	Rename pedal output flag from `is_valid` → `is_active`.
src/plugins/controller_synthetic_hands/synthetic_hands_plugin.cpp	Use controller tracked wrappers and guard reads on `.data`.
src/plugins/controller_synthetic_hands/README.md	Update documentation examples to use tracked wrappers (`.data`).
src/core/schema_tests/python/test_pedals.py	Update tests for `Generic3AxisPedalOutput.is_active`.
src/core/schema_tests/python/test_hand.py	Remove assertions/docs referencing removed `HandPoseT.is_active`.
src/core/schema_tests/python/test_full_body.py	Remove assertions/docs referencing removed `FullBodyPosePicoT.is_active`.
src/core/schema_tests/python/test_controller.py	Update `ControllerSnapshot` construction/expectations after removing `is_active`.
src/core/schema_tests/cpp/test_pedals.cpp	Update native tests for `is_active` and vtable constants.
src/core/schema_tests/cpp/test_hand.cpp	Remove vtable/type asserts and tests for removed `HandPose.is_active`.
src/core/schema_tests/cpp/test_full_body.cpp	Remove vtable/type asserts and tests for removed `FullBodyPosePico.is_active`.
src/core/schema_tests/cpp/test_controller.cpp	Update struct tests/serialization expectations after removing `is_active`.
src/core/schema/python/schema_init.py	Export newly added `*TrackedT` Python types.
src/core/schema/python/pedals_bindings.h	Bind `is_active` and add `Generic3AxisPedalOutputTrackedT`.
src/core/schema/python/oak_bindings.h	Add `FrameMetadataTrackedT` Python binding.
src/core/schema/python/locomotion_bindings.h	Add `LocomotionCommandTrackedT` Python binding.
src/core/schema/python/head_bindings.h	Add `HeadPoseTrackedT` Python binding.
src/core/schema/python/hand_bindings.h	Remove `HandPoseT.is_active` exposure and add `HandPoseTrackedT`.
src/core/schema/python/full_body_bindings.h	Remove `FullBodyPosePicoT.is_active` exposure and add `FullBodyPosePicoTrackedT`.
src/core/schema/python/controller_bindings.h	Remove `ControllerSnapshot.is_active` exposure and add `ControllerSnapshotTrackedT`.
src/core/schema/fbs/pedals.fbs	Rename `is_valid` → `is_active`; add `Generic3AxisPedalOutputTracked`.
src/core/schema/fbs/oak.fbs	Add `FrameMetadataTracked`.
src/core/schema/fbs/locomotion.fbs	Add `LocomotionCommandTracked`.
src/core/schema/fbs/head.fbs	Add `HeadPoseTracked`.
src/core/schema/fbs/hand.fbs	Remove `is_active`, renumber fields, and add `HandPoseTracked`.
src/core/schema/fbs/full_body.fbs	Remove `is_active`, renumber fields, and add `FullBodyPosePicoTracked`.
src/core/schema/fbs/controller.fbs	Remove `is_active` and add `ControllerSnapshotTracked`.
src/core/retargeting_engine_tests/python/test_sources.py	Update controller snapshot creation after signature change.
src/core/retargeting_engine/python/deviceio_source_nodes/hands_source.py	Accept `HandPoseT
src/core/retargeting_engine/python/deviceio_source_nodes/controllers_source.py	Accept `ControllerSnapshot
src/core/deviceio/python/deviceio_bindings.cpp	Update pybind query methods to return tracked wrappers.
src/core/deviceio/cpp/inc/deviceio/head_tracker.hpp	Change public API return type to `HeadPoseTrackedT`.
src/core/deviceio/cpp/inc/deviceio/hand_tracker.hpp	Change public API return type to `HandPoseTrackedT` and update internals.
src/core/deviceio/cpp/inc/deviceio/generic_3axis_pedal_tracker.hpp	Change public API return type to `Generic3AxisPedalOutputTrackedT`.
src/core/deviceio/cpp/inc/deviceio/full_body_tracker_pico.hpp	Change public API return type to `FullBodyPosePicoTrackedT`.
src/core/deviceio/cpp/inc/deviceio/frame_metadata_tracker_oak.hpp	Change public API return type to `FrameMetadataTrackedT`.
src/core/deviceio/cpp/inc/deviceio/controller_tracker.hpp	Change public API return type to `ControllerSnapshotTrackedT`.
src/core/deviceio/cpp/head_tracker.cpp	Implement tracked wrapper storage and updated serialization for head pose.
src/core/deviceio/cpp/hand_tracker.cpp	Implement tracked wrapper semantics (null on inactive) and updated serialization.
src/core/deviceio/cpp/generic_3axis_pedal_tracker.cpp	Wrap pedal output in tracked wrapper and reset `.data` when unavailable.
src/core/deviceio/cpp/full_body_tracker_pico.cpp	Wrap body pose in tracked wrapper and reset `.data` when inactive.
src/core/deviceio/cpp/frame_metadata_tracker_oak.cpp	Wrap metadata in tracked wrapper and update serialization behavior.
src/core/deviceio/cpp/controller_tracker.cpp	Wrap controller snapshots in tracked wrapper and reset `.data` when inactive.
examples/schemaio/pedal_pusher.cpp	Update pedal output field name to `is_active`.
examples/schemaio/pedal_printer.cpp	Update consumer to check `tracked.data` and dereference when present.
examples/schemaio/frame_metadata_printer.cpp	Update consumer to check `tracked.data` before printing.
examples/retargeting/python/sources_example.py	Update example to use tracked wrappers from DeviceIO.
examples/oxr/python/test_synthetic_hands.py	Update example to check `.data is not None` for hand activity.
examples/oxr/python/test_session_sharing.py	Update example to dereference tracked payloads.
examples/oxr/python/test_modular.py	Update modular test to use tracked wrappers and remove unused schema import.
examples/oxr/python/test_full_body_tracker.py	Update example to use tracked body pose wrapper.
examples/oxr/python/test_extensions.py	Update extension test to use tracked wrappers.
examples/oxr/python/test_controller_tracker.py	Update controller test to use tracked wrappers and guarded reads.
examples/oxr/python/modular_example.py	Update modular example to use tracked wrappers.
examples/oxr/cpp/oxr_simple_api_demo.cpp	Update C++ demo to use tracked wrappers for hands/head.
examples/oxr/cpp/oxr_session_sharing.cpp	Update session sharing demo to use tracked wrappers.
examples/lerobot/record.py	Update dataset recording example to use tracked wrappers and remove unused schema import.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

examples/retargeting/python/sources_example.py

src/core/schema/fbs/head.fbs

src/core/schema/fbs/controller.fbs

src/core/retargeting_engine/python/deviceio_source_nodes/hands_source.py

github-actions · 2026-02-24T19:44:56Z

Docs preview

Built docs for this PR are available at:

https://NVIDIA.github.io/IsaacTeleop/preview/pr-167/

(It may take a minute to appear after the workflow completes.)

github-actions · 2026-02-24T19:49:41Z

Docs preview

Built docs for this PR are available at:

https://NVIDIA.github.io/IsaacTeleop/preview/pr-167/

(It may take a minute to appear after the workflow completes.)

examples/oxr/python/test_full_body_tracker.py

src/core/deviceio/cpp/inc/deviceio/head_tracker.hpp

src/core/deviceio/cpp/controller_tracker.cpp

src/core/deviceio/cpp/generic_3axis_pedal_tracker.cpp

src/core/deviceio/cpp/head_tracker.cpp

src/core/retargeting_engine/python/deviceio_source_nodes/controllers_source.py

src/core/schema/fbs/pedals.fbs

examples/oxr/python/modular_example.py

examples/oxr/python/test_controller_tracker.py

src/core/schema/fbs/controller.fbs

examples/lerobot/record.py

examples/oxr/python/test_controller_tracker.py

nvddr force-pushed the devdeepr/createTrackedType branch from dd81037 to 1ba0936 Compare February 24, 2026 18:42

nvddr requested review from aristarkhovNV, Copilot and jiwenc-nv and removed request for aristarkhovNV and jiwenc-nv February 24, 2026 18:42

Copilot started reviewing on behalf of nvddr February 24, 2026 18:43 View session

Copilot AI reviewed Feb 24, 2026

View reviewed changes

nvddr force-pushed the devdeepr/createTrackedType branch from 1ba0936 to 7944724 Compare February 24, 2026 19:44

nvddr force-pushed the devdeepr/createTrackedType branch from 7944724 to f770695 Compare February 24, 2026 19:49

nvddr temporarily deployed to dev February 24, 2026 19:56 — with GitHub Actions Inactive

nvddr requested a review from jiwenc-nv February 24, 2026 20:08

nvddr temporarily deployed to dev February 24, 2026 20:31 — with GitHub Actions Inactive

aristarkhovNV reviewed Feb 24, 2026

View reviewed changes

jiwenc-nv approved these changes Feb 25, 2026

View reviewed changes

nvddr force-pushed the devdeepr/createRecordType branch 2 times, most recently from 7cda387 to 6f16255 Compare February 25, 2026 22:51

Base automatically changed from devdeepr/createRecordType to main February 25, 2026 23:49

nvddr force-pushed the devdeepr/createTrackedType branch from f770695 to 3fde3cf Compare February 26, 2026 00:22

nvddr temporarily deployed to dev February 26, 2026 00:25 — with GitHub Actions Inactive

nvddr had a problem deploying to dev February 27, 2026 10:27 — with GitHub Actions Error

nvddr temporarily deployed to dev February 27, 2026 10:31 — with GitHub Actions Inactive

Create tracked types with optional data to handle inactive input sources

2c38d46

nvddr force-pushed the devdeepr/createTrackedType branch from 1b06930 to 2c38d46 Compare February 27, 2026 10:36

nvddr temporarily deployed to dev February 27, 2026 10:36 — with GitHub Actions Inactive

nvddr had a problem deploying to dev February 27, 2026 10:36 — with GitHub Actions Failure

nvddr temporarily deployed to dev February 27, 2026 10:36 — with GitHub Actions Inactive

nvddr temporarily deployed to dev February 27, 2026 10:40 — with GitHub Actions Inactive

nvddr temporarily deployed to dev February 27, 2026 10:47 — with GitHub Actions Inactive

nvddr temporarily deployed to dev February 27, 2026 10:54 — with GitHub Actions Inactive

nvddr merged commit eedaa3e into main Feb 27, 2026
45 of 46 checks passed

nvddr deleted the devdeepr/createTrackedType branch February 27, 2026 10:59

Conversation

nvddr commented Feb 24, 2026 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

github-actions bot commented Feb 24, 2026

Docs preview

Uh oh!

github-actions bot commented Feb 24, 2026

Docs preview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Feb 24, 2026

Docs preview

Uh oh!

github-actions bot commented Feb 24, 2026

Docs preview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

nvddr commented Feb 24, 2026 •

edited by coderabbitai bot

Loading